Scalable Matrix-valued Kernel Learning and High-dimensional Nonlinear Causal Inference

We propose a general matrix-valued multiple kernel learning framework for highdimensional nonlinear multivariate regression problems. This framework allows a broad class of mixed norm regularizers, including those that induce sparsity, to be imposed on a dictionary of vector-valued Reproducing Kernel Hilbert Spaces [19]. We develop a highly scalable and eigendecomposition-free Block coordinate descent procedure that orchestrates two inexact solvers: a Conjugate Gradient (CG) based Sylvester equation solver for solving vector-valued Regularized Least Squares (RLS) problems, and a specialized Sparse approximate SDP solver [15] for learning output kernels. As an application of our framework, we show how high-dimensional causal inference tasks can be naturally cast as sparse function estimation problems within our framework, leading to novel nonlinear extensions of Grouped Graphical Granger Causality techniques. The algorithmic developments and extensive empirical studies are complemented by theoretical analyses in terms of Rademacher generalization bounds.